Skip to content

wasm: on-demand materialize + wrappers; non-fatal pinned init#5

Open
tiljrd wants to merge 5979 commits intotil/forking-capabilitiesfrom
devin/1755993593-wasm-on-demand-materialize
Open

wasm: on-demand materialize + wrappers; non-fatal pinned init#5
tiljrd wants to merge 5979 commits intotil/forking-capabilitiesfrom
devin/1755993593-wasm-on-demand-materialize

Conversation

@tiljrd
Copy link
Collaborator

@tiljrd tiljrd commented Aug 24, 2025

Enable mainnet WASM data availability on forked THORChain

Summary

This PR implements CosmWasm support for the THORChain forking client, allowing forked networks to access mainnet contract state and bytecode via gRPC. The implementation includes:

  • Forking client extensions to handle CosmWasm store keys (ContractInfo 0x02, CodeBytes 0x03, ContractStore 0x05) with mainnet gRPC fetching
  • Wasm materializer restoration to fetch bytecode from mainnet and write to local filesystem for contract execution
  • Query/Msg wrappers for on-demand contract materialization with mainnet fallback
  • Error handling adjustments to prevent RPC connection failures during wasm queries
  • Caching modifications to avoid negative cache poisoning during wasm key iteration

The changes enable smart contract queries like thornode query wasm contract-state smart <addr> '{"config":{}}' to work on forked networks by transparently fetching missing contract data from mainnet.

Review & Testing Checklist for Human

  • End-to-end execute testing: Import a test key and verify thornode tx wasm execute works without "not found" errors (the current test failed due to missing keyring key "mnc")
  • Key parsing validation: Test edge cases in contract address parsing logic in x/thorchain/forking/client.go lines 973-1009, especially the address reconstruction from raw bytes
  • Resource leak check: Verify gRPC connections to mainnet are properly closed and not leaking (multiple places create connections without clear pooling)
  • Error masking verification: Confirm that swallowing wasm remote errors in x/thorchain/forking/store.go lines 128-131 doesn't hide legitimate issues during development
  • Performance impact: Monitor query performance with negative caching disabled for wasm keys and consider re-enabling if iteration issues are resolved

Recommended test plan: Deploy the updated thornode image, run smart queries against known mainnet contracts, import a funded key, and execute a simple contract transaction to verify the full read/write flow works correctly.


Diagram

%%{ init : { "theme" : "default" }}%%
graph TB
    Client["thornode CLI<br/>wasm queries"]
    QueryWrapper["app/query_wasm_wrapper.go<br/>WasmQueryWrapper"]:::major-edit
    MsgWrapper["app/msg_wasm_wrapper.go<br/>WasmMsgWrapper"]:::major-edit
    Materializer["app/wasm_materialize.go<br/>materializeAndPinWasm"]:::major-edit
    ForkingStore["x/thorchain/forking/store.go<br/>forkingKVStore"]:::minor-edit
    ForkingClient["x/thorchain/forking/client.go<br/>remoteClient"]:::major-edit
    MainnetGRPC["grpc.thor.pfc.zone:443<br/>Mainnet gRPC"]:::context
    LocalFS["data/wasm/wasm/<br/>Local bytecode files"]:::context

    Client --> QueryWrapper
    Client --> MsgWrapper
    QueryWrapper --> Materializer
    MsgWrapper --> Materializer
    QueryWrapper --> ForkingStore
    MsgWrapper --> ForkingStore
    ForkingStore --> ForkingClient
    ForkingClient --> MainnetGRPC
    Materializer --> LocalFS
    QueryWrapper --> MainnetGRPC
    MsgWrapper --> MainnetGRPC

    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit
        L3["Context/No Edit"]:::context
    end

    classDef major-edit fill:#90EE90
    classDef minor-edit fill:#87CEEB
    classDef context fill:#FFFFFF
Loading

Notes

  • The forking client now handles complex CosmWasm key parsing with specific logic for different key types (ContractInfo, CodeBytes, ContractStore)
  • Error handling was adjusted to prevent RPC EOF errors, but this may mask legitimate issues
  • Negative caching is disabled for wasm keys to avoid cache poisoning during iteration
  • The mainnet gRPC endpoint is hardcoded in multiple places rather than being configurable
  • Full execute testing was blocked by keyring issues but smart queries work correctly

Session details: Requested by Til Jordan (@tiljrd) - https://app.devin.ai/sessions/24b3b308ea114effa590c47c761c616d

devin-ai-integration bot and others added 30 commits August 14, 2025 17:47
Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…ys with keeper format

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
… to prevent errors

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…e GET parity

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…regen gateway

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…l-get

forking(grpc): add single-pool GET support via gRPC; key normalization parity with keeper
…*} in HTTP bindings

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…d pulsar artifacts

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…rrower; robust key parsing for double-slash prefixes

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…ider, borrower; robust key parsing for double-slash prefixes"

This reverts commit 7cf4e3d.
…ed files by taking remote; will regenerate next

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…ing; route lp/ and loan/ keys

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…N/0XUPPERHEX); add saver single-GET

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…AIN/0x.. -> CHAIN.ETH and CHAIN.0X.. for single GETs

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…rower) to avoid list failures

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…case-insensitive address match

Co-Authored-By: francesco@bloctopus.io <ocsenarf@outlook.com>
…er with {asset=**}; regenerate gateway/types
This allows mimir transactions to bypass all ante handlers including
the SDK's DeductFeeDecorator, enabling them to work in test/forking
environments where validators may not have unbonded funds.
In forking mode, the bond module may have 0 balance. This change
checks the bond module balance before attempting to transfer fees
and skips the transfer if insufficient, allowing mimir transactions
to succeed in test/forking environments.
…kv-collision

app: deduplicate KVStoreKeys to fix 'acc' collision after authz integration
tiljrd and others added 30 commits September 8, 2025 10:52
…materialize' into devin/1755993593-wasm-on-demand-materialize

# Conflicts:
#	x/thorchain/forking/client.go
…minimal diagnostic to ensure WASMPERMISSIONLESS routing
… directly in forking mode; keep node vote and success logs
…CheckTx hangs when remote bank/auth reads fail
…ccount/sequence; fix Mimir tx hang in forking
…id mempool hangs; rely on DeliverTx for remote
…nblock account/sequence; keep others blocked
…rnode tx/query/keys can initialize with forking enabled
…nk-balances

Disable remote reads for bank balances in forking mode
…ance-hang

fix: prevent REST bank balance hang in forking mode (gateway timeout + broader height forwarding)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants